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Introduction 

New technologies and operational 
concept changes have been proposed for 
implementation in the National Airspace 
System (NAS). These changes include 
improved datalink (CPDLC) technologies for 
providing improved weather, traffic, Flight 
Object (FO) and navigation information to the 
pilot and controller, and new forms of 
automation for both the flight deck and air 
traffic management system (FAA, 1997). In 
addition, the way business is conducted in the 
NAS is under consideration. Increases in the 
discretion provided to pilots (and dispatchers in 
commercial airlines) are being contemplated in 
an effort to increase system capacity and 
flexibility. 

New concepts of operation (e.g., 
Collaborative Decision Making and Free 
Flight) allow for more control to be given to the 
cockpit or airline with correspondingly greater 
monitoring responsibilities on the ground. In 
addition, new technologies and displays make 
possible much greater information flow 
between the ground and the cockpit and also 
dramatic changes in the type of information 
provided. Designing to support these changes 
suggests two integrally linked questions: 

(1) What display technologies and 
information are needed to support 
desired changes in operational 
responsibilities? 

(2) How will the changes in information 
availability influence the negotiation 
process between the cockpit and the 
ground? 


Each of these proposed changes (both in 
technology and operational concept) will have 
a marked impact on the performance, workload 
and Situation Awareness (SA) of both pilots 
and controllers. Typically such changes are 
evaluated independently in terms of the effects 
of the proposed change on either pilot 
performance or ATC performance. It is 
proposed here, however, that in order to fully 
understand the effects of such changes, the 
joint | pilot/controller system must be 
considered. In particular, it will be shown that 
not oftly do changes in one part of the system 
affect the operators of that system, they may 
also induce shifts in the need for a shared 
understanding of the system between pilots and 
controllers (shared SA), and may induce shifts 
in the locus of control between these two 
agents as the degree of shared SA itself 
changes. 

The model in Figure 1 is presented as a 
framework for understanding the changes 
induced on the overall system by changes to 
one part of the system. Historically the 
controller possessed a high degree of control 
for directing the separation activities of aircraft. 
With this role, there was little need (or 
opportunity) for shared situation awareness 
between pilots and controllers as it was 
assumed that pilots would obey controller 
commands. Only the controller needed to have 
SA regarding this information. For activities 
performed primarily by the pilot (e.g. systems 
operations) it is assumed only the pilot needs to 
have this information, not the controller. 
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Figure 1. Shared SA and Locus of Control 
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As more information becomes available 
in the cockpit, the locus of control shifts to a 
middle ground. An example of this was the 
implementation of the TCAS collision 
avoidance system, where controllers had to deal 
with pilots interjecting unexpected actions (to 
the controllers) in response to TCAS alerts. In 
addition, TCAS was observed to change the 
negotiation process between controllers and 
pilots. With new operational concepts, such as 
an increase in self separation promised by some 
free flight advocates, it can be seen that the 
need for a shared common picture becomes 
much greater. Differences between controller 
SA and pilot SA may have substantial 
implications, resulting in problems such as 
increased negotiation or lack of immediate 
compliance to ATC commands by pilots. 
Therefore if pilots and controllers are going to 
have increasingly shared responsibilities for 
tasks (such as navigation, sequencing or 
separation activities) it will be increasingly 
important that they share a common picture of 
the environment. 

Analysis of Shared SA Requirements 

An analysis of shared goals and shared 
SA requirements between pilots and controllers 
in the NAS was conducted to examine this 
issue further. This was done by comparing an 
SA requirements analysis that was conducted 
for en route air traffic controllers (Endsley & 
Rodgers, 1994) with a similar analysis recently 
conducted for commercial airline pilots 
(Endsley, Farley, Jones, Midkiff, & Hansman, 
1998). These two sets of goals and require- 


ments were examined to form an understanding 
of (1) how these requirements overlap forming 
requirements for shared SA between pilots and 
controllers, (2) the mechanisms available to 
each party for acquiring SA on these factors in 
today’s system, and (3) the implications of 
proposed changes to the system for shared SA 
and locus of control in the NAS of the future. 

Example of Results 

The pilot and controller goal hierarchies 
were analyzed and structured in order to 
identify areas of common or conflicting interest 
between pilots and controllers. Figures 2a and 
2b depict the high-level goals of pilots and 
controllers, respectively. At these higher levels, 
the goal structures were highly parallel, and 
there was considerable overlap between the 
two. For example, both shared the need to 
avoid conflicts such as traffic and hazardous 
weather as well as the need to handle 
perturbations such as weather and emergencies. 

In examining the detailed pilot and 
controller SA requirements, considerable 
overlap was present. At the same time there 
were also many differences, some of which 
were quite subtle, even for goals which at least 
superficially appeared to be the same. 

For instance, pilots and controllers have 
a common goal to maintain aircraft separation. 
To the pilot, conformance to ATC clearances 
and instructions has been the major subgoal 
associated with achieving aircraft separation. 
Assuring that aircraft are conforming to 
clearances, on the other hand, is seen by 
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Figure 2a. Commercial Airline Pilot Top-Level Goal Hierarchy 
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Figure 2b. En Route Air Traffic Controller Top-Level Goal Hierarchy 


controllers as a separate subgoal; and one 
which has been found to be neglected, 
particularly as workload increases (Endsley & 
Rodgers, 1998). In addition, if controllers do 
detect non-conformance to a clearance, they 
undertake to ascertain the reason for the non- 
conformance, focusing on determining the 
validity of the data they are receiving. Even if it 
is determined that non-conformance exists, the 
controllers may only undertake to correct the 
aircraft if it poses a particular problem. 

Pilots on the other hand, did not 
represent these types of concerns in their goal 
breakouts. They focused on ascertaining any 
deviation between the ATC clearance and their 
current and projected flight parameters (in 


common with the controllers). To do this they 
needed several pieces of information that did 
not show up in the controllers’ breakout, 
including time available to meet the clearance 
and the magnitude and rate of change required 
to meet the clearance. This represents an 
“information gap” between ATC and pilots. 
Controllers rarely communicate time available 
for meeting a clearance (although sometimes 
they may in imprecise terms convey a sense of 
“urgency”). Pilots are left to figure out these 
higher-level SA requirements on their own. 
Suffering from a similar information gap, 
controllers may often have poor information on 
aircraft capabilities, weather and winds that are 
needed for making their own assessment of 
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whether an aircraft will be able to conform to a 
clearance as expected. 

Some interesting differences are also 
seen in the subgoals associated with 
determining whether aircraft are separated. 
While this subgoal clearly shows up for both 
ATC and the aircraft pilot, they appear to meet 
it in very different ways. The controllers very 
distinctly break their decisions up into a 
vertical separation and horizontal separation 
question, with SA requirements following these 
divisions. Furthermore, ATC compares aircraft 
separations to very specific limits, as 
appropriate for the particular airspace. No such 
distinctions were present for the aircraft pilots, 
however. They appeared to make a single 
decision regarding separation, and did not 
appear to use any fixed limits in assessing that 
separation. 

Finally, if it is determined that some 
action is needed to resolve an aircraft conflict, 
pilots and controllers also appear to approach 
this subgoal differently. While many of the 
decisions appear to be similar (e.g., which 
aircraft to maneuver, time available, which type 
of maneuver to use), ATC appears to adopt a 
much more strategic focus to its decisions 
while the aircraft pilot’s are far more tactical. 
For instance, ATC includes a specific 
consideration of the impact of a particular 
maneuver on air traffic and the projected 
number of changes that will be necessary to 
bring the aircraft back to its intended flight path 
and destination. They also specifically consider 
the capabilities of the aircraft and the pilots and 
how many previous actions (changes to 
clearance) had been given to a particular 
aircraft. They seek to optimize the flow of the 
traffic overall in making their decisions. The 
pilots' breakouts did not reveal these type of 
strategic considerations, but instead relied on 
more immediate tactical considerations, ATC 
clearances and the rules built into TCAS when 
deciding which aircraft would maneuver and 
when. 


Many of these differences are natural 
outcomes of the current division of responsi- 
bility between ATC and the cockpit, and of the 
information available to each. Even so, certain 
conflicts which exist today between these two 
decision makers can be traced directly back to 
the subtle differences in each party's interests 
and the different information that is available to 
them. 

One common example is re-route 
negotiation, wherein the pilot typically seeks to 
optimize his own route while the controller 
typically seeks to optimize the routes of many. 
The supporting information available to each is 
often different and sometimes exclusive. Given 
the potential for new technologies such as 
datalink to share information between the air 
and the ground, several issues arise: 

(1) How will the sharing of information 
between pilots and controllers affect 
their ability to recognize routing 
conflicts and opportunities? 

(2) Will it increase or decrease the amount 
of negotiation? 

(3) Will negotiations become more 
collaborative or more contentious? 

Simulator-Based Experiment 

Based on the comparative SA 
requirements analysis, a part-task simulator 
experiment was conducted to investigate the 
effect of sharing weather and traffic 
information (via air-ground datalink) between 
pilots and controllers in negotiating route 
amendments. Weather and traffic were chosen 
as the key conflict categories because pilots and 
controllers reported in the SA requirements 
analyses (see Figures 2a & b) that weather and 
traffic are often the reason for re-routing and 
typically impose constraints on the available 
alternatives. Of particular interest in this 
experiment were indications of each subject's 
recognition of the other's constraints, his antici- 
pation of the other’s needs and/or preferences, 
his willingness to comply/cooperate, and his 
persistence in pursuing an alternate solution. 


4 


Experimental Design 

The experiment paired an air transport 
pilot subject with an en route air traffic 
controller subject in a real-time simulated air 
traffic environment under present-day air traffic 
control procedures. Test scenarios featured 
traffic and weather elements specifically 
designed to create testable responses, a 
performance-based measure of situation 
awareness (Pritchett, Hansman, & Johnson, 
1996). Testable response scenarios incorporate 
a hazard element (e.g., an intruder aircraft, a 
weather cell) that requires the subject to take 
action, provided s/he is aware of the situation. 
An appropriate action taken by the subject 
indicates situation awareness; inaction indicates 
a lack of situation awareness. Subjects 
interacted within the simulation environment to 
resolve the traffic and weather conflicts. The 
availability of shared traffic and weather 
displays was manipulated as the independent 
variable as shown in Table 1 . 


Table 1. Test Matrix 


Configuration 

Traffic 

information 

Weather 

information 

Without 

datalink 

Controller 

only 

Pilot 

°niy 

With datalink 

Shared 

Shared 


Because traffic displays are standard air 
traffic control equipment, the controller was 
always provided with traffic information, even 
in the non-datalinked configuration. Similarly, 
because weather radar is standard equipment in 
jet aircraft, the pilot was always provided with 
weather information, even in the non- 
datalinked configuration. 

Figure 3 shows the cockpit and ATC 
displays provided to the test subjects in the 
datalinked configuration, featuring both 
weather and traffic information on each. 
Weather information was depicted by six-color 
NEXRAD reflectivity images. Air traffic 
information was depicted on the controller's 
plan view display in a formal consistent with 



Figure 3. Cockpit map display and ATC plan 
view display including weather and traffic 


current en route ATC equipment and included 
aircraft position, call sign, track, altitude, and 
ground speed. The pilot's cockpit display of 
traffic information (CDTI) depicted the 
position, call sign, track, relative altitude, and 
relative ground speed of all aircraft within 40 
miles and 2600 feet in altitude. 

Each pilot-controller subject pair 
performed three test scenarios two times, once 
with shared traffic and weather displays (i.e., 
"with datalink") and once without (i.e., 
"without datalink"). All scenarios took place in 
a high-level sector in Indianapolis Center 
airspace. Each scenario ran approximately ten 
minutes and featured between 12 and 18 
aircraft transitioning the sector in the presence 
of convective weather activity. Those aircraft 
not piloted by the subject pilot were controlled 
by a pseudo-pilot, who also interacted with the 
subject controller and subject pilot via radio 
communication. Certain elements of each 
repeated scenario were changed (e.g., aircraft 
call signs, trajectories of non-factor traffic, etc.) 
in order to make the second run appear unique. 

Pilot and controller situation awareness 
was measured using the testable response 
method. Radio communications were recorded 
and coded using a methodology adapted from 
Foushee, Lauber, Baetge, & Acomb (1986). 
Workload measurements were taken based on 
the NASA Taskload Index (TLX) (Hart and 
Stave land, 1988). 
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In order to observe pilot-controller 
interaction in a real-time, complex workload 
environment, MIT's distributed, interactive, 
multi-agent simulation facility was used 
(Amonlirdviman, et. al., 1998). The facility 
includes a part-task advanced cockpit 
simulator, a part-task en route controller 
workstation, a multi-aircraft pseudo-pilot 
station, and live voice communications 
between them, creating a real-time interactive 
air traffic environment. 

Preliminary Results 

Two pilot-controller teams have 
performed the experiment to date. Both 
controller subjects were Full Performance 
Level (FPL) ATC Specialists with an average 
of 10.5 years of experience, currently working 
the radar position at an Air Route Traffic 
Control Center (ARTCC). Both pilot subjects 
were jet transport pilots with an average of 
12,850 hours. 

Due to the small sample size, it is 
emphasized that the results presented here are 
preliminary. Good scientific judgment should 
be exercised before attempting to apply these 
results generally. 

Situation Awareness 

Each test scenario included one 

weather-related testable response condition and 
one traffic-related testable response condition. 
Both the pilot and controller were monitored 
for their awareness of each testable response 
condition. 

Figure 4 summarizes their situation 
awareness with respect to weather. Pilots, 
having the benefit of the weather display for all 
test scenarios, demonstrated awareness of all of 
the weather-related testable response 
conditions. Controllers, without the benefit of a 
weather display in the non-datalinked 

configuration, demonstrated awareness of only 
half of the weather-related testable response 
conditions. When provided a weather display, 
however, controllers demonstrated awareness 
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Figure 4. Pilot and controller awareness 
of weather-related testable response 
conditions. 

of all of the weather-related testable response 
conditions. 

Figure 5 summarizes pilot and 
controller situation awareness with respect to 
traffic. Pilots, without the benefit of a traffic 
display in the non-datalinked configuration, did 
not demonstrate awareness of any of the traffic- 
related testable response conditions. In many 
cases, the controllers recognized the traffic 
conflict before it became a significant threat to 
the pilot and would either advise the pilot of the 
traffic or vector him accordingly. In such cases, 
the pilot's testable response result was labeled 
"ambiguous". Controllers, having the benefit of 
their plan view traffic display for all test 
scenarios, demonstrated a high level of 
awareness of the traffic-related testable 
response conditions. In some cases, controller- 
issued clearance amendments inadvertently 
resolved the traffic-related testable response 
conditions before they arose; such cases were 
labeled "ambiguous" with respect to controller 
situation awareness. 

These results indicate that controller SA 
with respect to weather improves with the 
addition of a weather overlay to their plan view 
display. Similarly, the results suggest that pilot 
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Figure 5. Pilot and controller awareness 
of traffic- related testable response 
conditions. 

SA with respect to traffic improves with the 
addition of a CDTI. 

Pilot and controller workload was 
measured using the NASA TLX methodology. 
In general, workload was not affected in any 
systemic way, either individually or in a team 
sense. 

In the twelve test scenarios completed 
thus far in the study, two operational errors 1 
were observed, both occurring in the non- 
datalinked configuration. It is important to note 
that the test scenarios were challenging by 
design. Controllers were operating an air traffic 
sector other than their usual "home" sector and 
did not have the benefit of a conflict alert 
function or a D-side controller to assist them. 
However, the fact that both operational errors 
occurred in the non-datalinked environment 
does suggest that shared information may be 
beneficial in helping the controller maintain 
awareness with regard to separation issues. 

Communication and Negotiation 

All radio communication was recorded, 
coded by category and topic, and analyzed. 
Figure 6 illustrates how the transactions 


conducted over the voice channel changed with 
the introduction of the datalink. As shown at 
the left, the number of transactions between the 
pilot and controller decreased when the 
datalink was introduced. However, the number 
of transactions for negotiating re-route 
clearances increased slightly, and the number 
of other transactions (including traffic 

advisories, ride reports, etc.) decreased. 
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Figure 6. Voice communication transactions by 
topic 

Figure 7 illustrates how the character of 
radio communications changed with the 
introduction of the datalink. The total number 
of transmissions, defined as "mic on" to "mic 
off, increased slightly (from 87 transmissions 
to 93) with the sharing of information via 
datalink. In analyzing these transmissions in 
more detail, several trends were observed. The 
number of pilot requests went up, consistent 
with the increase in re-route negotiation noted 
earlier. When the weather and traffic 
information was shared between the pilot and 
controller, there were four instances in which 
the pilot or controller voluntarily suggested a 
specific route amendment to the other party. 
This behavior was not evident in the non- 
datalinked configuration. Finally, controllers 
were much more proactive in providing 
weather advisories to pilots when they had the 
weather information overlay. 


1 An operational error is defined as lateral separation of less 
than five miles and vertical separation of less than 1000 feet. 
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Figure 7. Voice communication transmissions 
by category 


Conclusions 

The comparison of pilots' and 
controllers' goals and SA information require- 
ments revealed many parallels at the higher 
levels. At the more detailed, lower levels, 
however, subtle underlying differences were 
identified that lend insight into their decisions. 

The preliminary experimental results 
indicate that situation awareness was improved 
when information was shared between the 
cockpit and the ground. Pilots' traffic situation 
awareness improved when traffic information 
was displayed; controllers’ weather situation 
awareness improved when weather information 
was displayed. Operational errors were reduced 
in the process. 

Pilots and controllers also exhibited 
behavioral changes. When information was 
shared, there were fewer requests for 
information, and controllers provided more 
frequent weather advisories to pilots. Pilots and 
controllers were more likely to suggest 
preferred route amendment alternatives to their 
counterparts, and re-route negotiation increased 
in the process. 

Based on these preliminary results, it 
appears that shared information between flight 
crews and air traffic controllers may be 
beneficial in terms of improved situation 
awareness and the behaviors it seems to foster. 
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